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Integrated  Learning  of  Words  and  their  Underlying  Concepts 

Introduction 

Previous  computational  models  of  the  acquisition  of  word  meaning  [Berwick83.  Granger77. 
Self ridge82]  have  assumed  existing  knowledge  of  the  concept  underlying  the  word  to  be  learned.  In 
these  models,  word  learning  is  a  process  of  using  surrounding  context  to  establish  an  identification 
between  a  new  lexical  item  and  a  known  concept.  However,  new  words  are  not  always  encoun¬ 
tered  as  labels  for  known  concepts.  When  encountering  a  new  concept  in  natural  language  text  or 
discourse,  it  is  quite  likely  that  one  will  also  come  across  unknown  words  which  refer  to  various 
aspects  of  the  new  concept.  A  word  learning  model  which  requires  prior  knowledge  of  the  under¬ 
lying  concept  will  be  unable  to  acquire  even  provisional  meanings  for  such  words. 

Developmental  studies  suggest  that  the  learning  of  words  and  their  underlying  concepts  fre¬ 
quently  occurs  concurrently.  Experiments  by  Gopnik  and  Meltzoif  revealed  that  children’s  acquisi¬ 
tion  of  ‘disappearance*  words  occurred  at  about  the  same  time  they  learned  to  solve  object- 
permanence  tasks  involving  invisible  displacements  [Gopnik86].  From  this  data,  they  concluded 
that  learning  may  often  involve  " concurrent  cognitive  and  semantic  developments,  rather  than 
involving  cognitive  prerequisites  for  semantic  developments."  Bowerman  [BowermanSO]  and  Kuc- 
za j  [k.uczaj82]  have  also  used  developmental  data  to  argue  for  an  interactive  approach  to  language 
and  concept  acqutsiiion. 

This  paper  describes  an  integrated  computational  model  of  the  acquisition  of  word  meanings 
and  their  underlying  concepts.  This  approach  was  developed  in  an  attempt  to  add  word  learning 
abilities  to  the  GENESIS  explanation -based  schema  acquisition  system  [Mooney85].  From  a  single 
natural  language  narrative,  the  current  GENESIS  system  is  able  to  acquire  a  new  schema  as  well  as 
provisional  meanings  for  several  schema  related  words. 

Ajb  Overview  of  Schema  Acquisition  in  GENESIS 

GENESIS  [\1ooney85]  is  an  explanation-based  learning  system  [DeJong86  MitcbellSh]  which 
learns  a  plan  schema  from  a  single  instance  by  determining  why  a  particular  sequence  ol  actions 
observed  in  a  specific  narrative  allowed  the  actors  to  achieve  their  goals  During  the  understanding 
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process.  GENESIS  attempts  to  construct  explanations  for  characters'  actions  in  terms  of  the  goals 
their  actions  were  meant  to  achieve.  This  process  involves  plan  and  script-based  understanding 
mechanisms  like  those  employed  by  previous  narrative  processing  systems  [Schank8l].  When  the 
system  observes  that  a  character  has  achieved  an  interesting  goal  in  a  novel  way.  it  generalizes  the 
composition  of  actions  the  character  used  to  achieve  this  goal  into  a  new  schema.  The  generaliza¬ 
tion  process  (described  in  [Mooney86])  consists  of  an  analysis  of  the  causal  model  of  the  narrative 
which  removes  unnecessary  details  while  maintaining  the  validity  of  the  explanation.  The  result- 
mg  generalized  set  of  actions  is  then  stored  as  a  new  schema  and  used  by  the  system  to  correctly 
process  narratives  which  were  previously  beyond  its  capabilities. 

In  [Ahn87].  experimental  evidence  is  presented  which  indicates  that,  like  GENESIS,  people  can 
also  a  acquire  a  schema  by  generalizing  the  explanation  of  a  single  narrative.  After  reading  one 
specific  instance  of  a  novel  plan,  subjects  were  able  to  describe  the  underlying  schema  in  abstract 
terms,  generate  a  different  instance,  and  correctly  answer  questions  about  the  general  schema. 

An  Example  of  Learning  Words  with  their  Concepts 

This  section  presents  an  example  of  GENESIS'  ability  to  learn  word  meanings  as  well  as  their 
corresponding  concepts  from  a  single  example.  First  the  system  is  given  the  following  kidnapping 
story  At  this  point.  GENESIS  has  schemata  for  threatening,  capturing,  making  bargains,  and  a 
number  of  other  actions  as  well  as  definitions  for  many  words:  however,  it  does  not  have  a  schema 
for  kidnapping-for-ransom  nor  definitions  for  the  words  "kidnap*,  "kidnapper*,  or  "ransom" 

Story  1 

Fred  it  Marv't  father  and  it  a  millionaire  John  approached  Mary  and  pointed  a  gun  at  her  She 
vat  wearing  blue  jeant.  He  lold  her  if  the  did  not  gel  in  hit  car  then  he  would  shooi  her  He  drove 
her  10  hit  hotel  and  locked  her  in  hit  room.  John  called  Fred  and  lold  him  John  wat  holding  Marv 
captive  John  told  Fred  if  Fred  gave  him  $250000  at  Trenot  then  John  would  releaae  Mary  Fred 
paid  him  the  iimcn  and  the  kidnapper  released  Marv  \  alenr  it  Fred  t  wife  and  he  lold  her  ihai 
tomeone  had  kidnapped  Marv 

From  this  single  instance.  GENESIS  learns  a  general  schema  for  kidnapping-f or-ransom  'which  it 
•.alls  CaptureBargain  based  on  the  two  main  actions  which  compose  it  >  as  well  as  preliminary 
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definitions  for  "kidnapper”,  "ransom",  and  "kidnap”.  A  paraphrase  of  the  CaptureBargain  schema  is 
shown  below: 

CaptureBargain(?x97.?a52,?bl  l.?c4.?y  15.7119) 

?bll  is  a  person.  7c4  is  a  location.  7x97  is  a  character.  ?bll  is  free.  7x97  captures  ?bll.  7aS2  it  a 
character.  7x97  contacts  ?a32  and  tells  it  that  ?bll  is  7x97's  captive.  7yl5  is  a  valuable.  7x97 
wants  to  have  7yl5  more  than  it  wants  7bll  to  be  7x97*s  captive.  ?aS2  has  a  positive  relationship 
with  7bl  1.  7a52  has  ?ylS.  7x97  and  7a52  carry  out  a  bargain  in  which  7x97  releases  7bl  1  and  7a52 
gives  7x97  7ylS  at  7119. 

The  provisional  definition  learned  for  "kidnap"  is  an  action  describing  an  instance  of  CaptureBargain 
where  the  subject  is  the  actor  (7x97)  and  the  direct  object  is  the  person  he  captures  and  then 
releases  in  exchange  for  payment  (Tbll).  The  definition  conjectured  for  “kidnapper*  is  a  person 
filling  the  ap tor  role  of  the  new  schema  (7x97)  and  the  definition  for  "ransom"  is  a  valuable  item 
given  to  this  actor  (7yl5).  These  definitions  do  not  exactly  match  the  standard  dictionary 
definitions  of  these  words,  but  they  are  reasonable  approximations  given  their  use  in  this  one  exam¬ 
ple.  The  lexical  and  schematic  knowledge  acquired  from  this  example  enables  the  system  to  subse¬ 
quently  explain  the  following  stories  as  instances  of  its  new  CaptureBargain  schema. 

Story  2 

Ted  is  Alice's  husband.  A  kidnapper  took  Alice  into  a  room.  Bob  got  $75000  and  released  Alice. 

Story  3 

Ted  is  Alice's  husband.  John  took  Alice  into  a  room.  Ted  paid  John  the  ransom  and  John  released 
Alice 

Story  4 

Steve  kidnapped  Valerie.  Mike  was  Valerie's  father  and  paid  Steve  $30000. 

Prior  to  learning.  GENESIS  could  not  construct  explanations  for  any  of  these  stones  since  each  one 
requires  both  knowledge  of  the  schema  and  a  definition  for  the  appropriate  kidnap-related  word. 


Integrated  Learning  of  Words  and  their  Underlying  Concepts 


How  GENESIS  Learns  Word  Meanings 

Since  the  processes  for  learning  role  labels  such  as  "kidnapper"  and  "ransom"  and  that  for 
learning  schema  labels  such  as  "kidnap"  are  somewhat  different,  each  of  these  procedures  will  be 
discussed  separately. 

Learning  Role  Labels 

The  procedure  used  to  learn  role  labels  is  similar  to  the  technique  used  by  the  FOUL-UP  sys¬ 
tem  [Granger77]  except  that  it  is  integrated  with  the  schema  learning  and  schema  activation 
processes.  When  the  parser  (a  modified  version  of  McDYPAR  [Dyer83])  encounters  an  unknown 
word  when  there  is  an  expectation  for  a  noun  of  some  class,  a  dummy  variable  is  created,  anno¬ 
tated  with  the  unknown  word,  and  allowed  to  fill  the  expectation.  For  example,  the  phrase  "Fred 
paid  him  the  ransom"  in  Story  1  is  parsed  into  the  assertion:  Atrans(Personl.?xl.Person3)  where 
?xl  is  marked  with  the  fact  that  it  came  from  the  unknown  word  "ransom."  If  an  input  pattern 
with  an  unknown-word  variable  like  ?xl  matches  a  pattern  expected  by  currently  active  schema, 
then  a  provisional  definition  for  the  word  is  made  based  on  the  constraints  on  the  schema  role 
which  matches  the  variable.  For  example,  in  the  "ransom"  case,  the  previous  sentence  in  Story  1 
suggested  a  Bargain  schema  between  John  and  Fred  and  set  up  expectations  for  the  two  proposed 
actions.  Since  "Fred  paid  him  the  ransom"  matches  the  expected  action  "Fred  gives  John  the 
$250000."  and  "ransom"  fills  the  role  of  the  item  whose  possession  is  transferred,  an  initial 
definition  is  made  for  "ransom"  stating  that  it  is  a  physical  object  whose  possession  is  transferred 
during  a  Bargain. 

However,  this  is  not  the  final  definition  created  for  "ransom*  since  an  additional  process  is  per¬ 
formed  when  a  new  schema  is  learned.  Each  of  the  sub-actions  composing  a  new  schema  is  checked 
for  roles  which  are  filled  by  unknown-word  variables  or  which  were  previously  matched  to  such  a 
variable  resulting  in  an  initial  definition.  In  either  case,  a  new  definition  is  created  for  the  unk¬ 
nown  word  based  on  the  role  it  fills  in  the  learned  schema  and  the  schema  constraints  on  this  role. 
Consequently,  when  the  CaptureBargain  schema  is  subsequently  recognized  and  learned  from  Story 
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1.  it  causes  "ransom"  to  be  redefined  as  a  valuable  item  whose  possession  is  transferred  to  the  actor 
in  a  CaptureBargain  schema  (?yl5).  The  rationale  for  having  learned  schemata  take  precedence 
when  defining  such  words  is  that  a  learned  schema  represents  a  new  situation  and  therefore  new 
words  are  assumed  more  likely  to  be  directly  associated  with  it  than  with  an  existing  schema  like 
Bargain. 

The  provisional  definition  for  a  role  label  like  "ransom"  contains  two  parts.  The  first  is  a  set 
of  constraints  on  the  object  itself,  such  as  an  assertion  that  it  be  Valuable.  The  second  part  is  a 
suggestion  of  the  schema  of  which  it  is  a  role.  The  fact  that  a  rble  label  definition  can  suggest  a 
relevant  schema  allows  GENESIS  to  use  the  definition  to  construct  explanations  for  narratives 
which  it  otherwise  would  not  understand.  When  the  word  "ransom"  is  subsequently  encountered 
in  Story  3.  it  suggests  that  the  CaptureBargain  schema  might  be  relevant.  This  schema  is  then  used 
in  a  top-down  fashion  to  construct  an  explanation  for  the  text.  Since  no  other  piece  of  information 
suggests  CaptureBargain.  the  learned  definition  for  "ransom"  is  crucial  in  understanding  this  story. 

However,  most  words  are  not  role  labels  like  "kidnapper"  and  "ransom."  For  example,  con¬ 
sider  replacing  the  word  "ransom"  in  Story  1  with  the  word  "moolah."  Since  the  word  "moolah"  is 
unknown.  GENESIS  gives  it  a  definition  identical  to  the  one  it  learned  for  "ransom."  In  order  to  be 
able  to  recover  from  such  mistakes,  the  system  monitors  the  schemata  suggested  by  newly  learned 
words.  If  a  new  word  subsequently  suggests  a  schema  which  does  not  explain  any  future  inputs, 
the  suggestion  is  removed.  Consequently,  after  receiving  a  murder-for-inheritance  story  in  which 
the  word  "moolah"  is  used,  "moolah"  ceases  to  suggest  CaptureBargain. 

Learning  Schema  Labels 

Learning  meanings  for  verbs  which  refer  to  entire  plan  schemata  is  a  more  difficult  task  since 
the  relevant  context  is  potentially  much  broader.  A  sentence  such  as  "John  robbed  the  store"  may 
be  used  to  introduce  a  long  piece  of  text  elaborating  the  situation,  to  succinctly  summarize  a  previ¬ 
ous  piece  of  text,  or  to  simply  refer  to  a  single  action  in  an  even  larger  plan.  A  number  of  heuris¬ 
tics  have  been  developed  which  allow  a  reasonable  guess  to  be  made  regarding  the  reference  of  such 
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unknown  verbs.  The  following  one  is  used  to  resolve  the  meaning  of  "kidnap"  as  used  in  Story  1. 

If  one  character  informs  another  that  some  action  occurred  and  a  schema  whose  actor  is  the 
same  as  this  action's  was  recently  acquired  from  the  narrative,  and  this  schema  also  has 
roles  filled  by  the  speaker  and  any  direct  and  indirect  objects  of  the  action,  then  assume 
the  speaker  is  summarizing  the  event  and  that  the  unknown  act  refers  to  the  new  schema. 

Specifically,  since  Fred  tells  his  wife  that  "someone  kidnapped  Mary"  and  both  him  and  Mary  were 
participants  in  the  just  completed  CaptureBargain  schema,  GENESIS  assumes  "kidnap"  refers  to 
CaptureBargain  (i.e.  one  can  summarize  the  schema  by  saying:  ?x97  kidnapped  ?bll).  An  appropri¬ 
ate  definition  is  then  created  for  "kidnap"  and  can  be  used  to  help  understand  Story  4.  In  this  nar¬ 
rative,  "Steve  kidnapped  Valerie"  is  interpreted  as  describing  an  instance  of  CaptureBargain  in 
which  Steve  is  the  actor  and  Valerie  is  the  victim.  The  assertions  that  "Mike  is  Valerie's  father" 
and  "Mike  paid  Steve  $30000"  are  then  understood  as  parts  of  the  expansion  of  this  instance  of  Cap¬ 
tureBargain.  Another  heuristic  involves  the  mentioning  of  an  unknown  action  followed  by  an  ela¬ 
boration  which  describes  a  novel  schema. 

Conclusions 

Unlike  previous  approaches  to  the  acquisition  of  word  meanings,  the  present  approach  does 
not  assume  prior  knowledge  of  the  underlying  concepts.  Learning  definitions  for  new  words  is 
integrated  with  an  explanation-based  concept  learning  mechanism.  This  allows  the  system  to  learn 
concepts  and  word  meanings  concurrently,  which  is  a  phenomenon  which  has  been  observed  in 
developmental  studies.  However,  this  paper  reports  only  preliminary  work  in  the  area  of 
integrated  word  learning.  There  are  many  problems  which  still  need  to  be  addressed.  A  few  of 
these  are  listed  below. 

(1)  The  procedure  for  removing  schema  suggestions  from  new  definitions  is  too  strict.  One 
counter-example  should  not  eliminate  a  suggestion  and  repeated  usefulness  of  a  suggestion 
should  make  it  resistant  to  elimination. 

(2)  Morphology  of  unknown  words  should  be  considered.  A  "kidnapper"  is  clearly  the  actor  of  a 
"kidnapping." 
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(3)  More  and  better  heuristics  are  needed  for  determining  whether  a  word  might  be  a  schema  label 
and  to  what  schema  it  might  refer. 

(4)  Only  role  labels  and  schema  labels  are  considered.  Many  words  do  not  fall  into  either  of  these 
two  categories. 

(5)  Only  integration  with  explanation-based  learning  is  considered.  Integration  with  similarity- 
based  learning  [Dietterich83]  should  also  be  examined. 

Nevertheless,  the  current  work  demonstrates  the  feasibility  and  usefulness  of  integrating  the 
acquisition  of  word  meanings  and  concepts.  Further  research  is  needed  in  both  AI  and  psychology 
to  explore  the  potential  symbiotic  relationship  between  language  and  concept  acquisition. 
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